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ABSTRACT 

An effective integration of human abilities and limitations is crucial to the success of all NASA missions. 
The Integrated Human Factors Toolkit facilitates this integration by assisting system designers and 
analysts to select the human factors tools that are most appropriate for the needs of each project. The HF 
Toolkit contains information about a broad variety of human factors tools addressing human requirements 
in the physical, information processing, and human reliability domains. Analysis of each tool includes 
consideration of the most appropriate design stage, the amount of expertise in human factors that is 
required, the amount of experience with the tool and the target job tasks that are needed, and other factors 
that are critical for successful use of the tool. The benefits of the Toolkit include improved safety, 
reliability and effectiveness of NASA systems throughout the agency. This report outlines the initial 
stages of development for the Integrated Human Factors Toolkit. 
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DEVELOPMENT OF AN INTEGRATED HUMAN FACTORS TOOLKIT 


Marc L. Resnick, Ph.D. 


1. INTRODUCTION 

Human Factors is the discipline devoted to optimizing the integration of human abilities and limitations 
into systems design to achieve safe, reliable, and effective performance. All of these objectives are 
critical for NASA to achieve its mission. Human factors is thus an integral part of the design and 
development process of NASA systems. However, in order to achieve successful integration of human 
factors considerations, they must be integrated throughout the design process. Systems engineers who do 
not have expertise in human factors must recognize when human factors is needed, how it should be 
applied, and when specialized expertise in human factors must be recruited. 

This report describes the initial stages of development of an Integrated Human Factors Toolkit (IHFT). 
The IHFT is proposed as a utility to assist systems engineers in the identification of human factors needs 
in a project, the selection of the most appropriate human factors tool, and the implementation of that tool. 
Optimal tool selection depends on the: 

• requirements of the system under development, 

• current stage of system completion, 

• resources available, and 

• expertise of the project team. 

Once a tool is selected, the toolkit also provides advice on how the tool can be implemented to maximize 
its effectiveness. By facilitating the integration of human factors needs into the project and improving 
tool use, the IHFT can improve the safety, reliability and effectiveness of the resulting system designs. It 
also can reduce the amount of time required to implement human factors, thus improving the productivity 
of the design process. 

2. HUMAN REQUIREMENTS 

Human performance can be divided into several components, mcluding physical performance, 
information processing, and human reliability. Physical performance refers to the safety and speed at 
which physical tasks can be completed. Safety is important for several reasons. Severe injury removes 
employees from the workforce despite investments in training and experience throughout their years of 
service [1], Injury can result from a single case of overexertion [2] or an accumulation of musculoskeletal 
damage from repeated movements [3], The development of injury depends on a complicated interaction 
of multiple workplace, physical, and organizational factors that must be considered in detail to minimize 
the risk of injury [4], Models of physical human capabilities have been created and combined with 
toolkits for biomechanical evaluation to create systems that can predict capabilities such as reach, fit, 
fatigue, and injury risk [5], 

Speed is also critical Not only does speed have a direct effect on productivity, but it can also have more 
insidious effects on safety and reliability. Fatigue increases the likelihood of injury [1], in part because of 
an increased probability of using incorrect postures and in part because of reduced integrity of fatigued 
muscles. Fatigue can also reduce coordination, increasing the probability of error in tasks requiring 
accurate manual control, such as interacting with a console interface. 
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Information Processing refers to the ability of a system user to perceive all of the information needed to 
complete job-related tasks, interpret and analyze that information, and make appropriate decisions, all 
while maintaining an acceptable level of mental workload. There are several computational models that 
have been developed to predict human information processing performance. The Man Machine 
Integration Design and Analysis System (MIDAS) was created by a joint US Army - NASA Ames team 
[6], MIDAS includes models of basic IP processes such as perception, memory, decision making, motor 
behavior, routine action, and skill, but does not include higher-level processes such as problem solving, 
learning, language, motivation, emotion, or imagining. The model can be used to predict performance 
times and levels of mental workload, as well as evaluating competing design alternatives [7], MIDAS has 
been validated in many situations such as air traffic control with encouraging results [8], There are also 
several commercially available modeling systems, such as the MicroSaint family from Micro Analysis 
and Design. 

Some of the landmark techniques in Human Reliability Assessment (HRA) are described by Kirwan [9], 
HRA can be decomposed into several consecutive stages. First, the tasks required to complete a job 
successfully must be analyzed in sufficient detail to accomplish the project goals. This level of detail 
depends on the stage of system development and the focus of the analysis. Once the tasks are analyzed, 
possible errors must be identified, represented, assessed, quantified, and reduced. Techniques for 
achieving each of these stages have been developed. Many of these are based on a framework developed 
by Rasmussen [10], which divides human behaviors into three types: 

• skill-based: automatic behavior based on well-learned procedures requiring little attention for 
activation, 

• rule-based: procedural behavior that can be followed based on a deterministic set of rules, and 

• knowledge-based: event-specific behavior that depends on context and involves problem solving 
processes. 

3. COMPILATION OF THE TOOL LIST 

It is essential that a general use Human Factors Toolkit contain all of the tools that should be considered 
for use by engineers and designers across NASA. This includes both those tools that best satisfy the 
expected needs of the engineers and designers to efficiently and effectively conduct their analyses as well 
as those that are preferred because of previous use and training. In order to insure that the tool 
compilation meets these needs, a comprehensive search for human factors tools from a variety of domains 
was conducted. Several domains were considered, including a variety of Department of Defense (DOD) 
collections, academic sources, literature review, surveys of current NASA employees and contractors, and 
personal communications (see Table 3-1). Several sources provided analysis of the tools in their 
inventories, either conducted by the tool developer or by independent observers. 

3.1 Sources 

DoD - The Department of Defense maintains an evolving compilation of human factors tools through the 
Defense Technical Information Center (DTIC). Originally developed by the Human Factors Engineering 
Technical Advisory Group in 1994, it continues to be updated as new tools are identified. Tool reviews 
include basic information on appropriate uses, hardware requirements, processing procedures, output 
formats, contact information, and validation references. 
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ONR - The Office of Naval Research Science and Technology Office established a Manning 
Affordability Initiative to prepare for expected reductions in manpower for naval systems. By 
encouraging the integration of human factors into the systems engineering process, they hoped to 
accommodate the reduced manpower without a corresponding reduction in system effectiveness. The tool 
analyses include descriptions of the tool capabilities, expertise requirements, hardware specifications, and 
constraints. There are also descriptions of how each tool fits within the Systems Engineering process and 
suggestions for how each tool complements and is complemented by other human factors and systems 
engineering tools. 

Eurocontrol: Eurocontrol is a European organization dedicated to the safety of air navigation throughout 
that continent. As part of its mission, it has created a human factors group to investigate human factors 
issues in future air traffic management systems development (HIFA). HLFA has created an interactive 
program to assist in the selection of human factors tools throughout the systems engineering design 
process for managers, designers and human factors practitioners. Brief descriptions and contact 
information are presented for each tool. 

NASA: The National Aeronautics and Space Administration Office of Safety and Mission Assurance has 
compiled a preliminary list of tools used by human factors practitioners throughout the agency. Tools are 
added to the list by practitioners, along with references and links to more information from other sources. 
No analysis of each tool is provided. Additional NASA tools were identified through surveys at Kennedy 
Space Center of NASA employees and contractors from Boeing, Lockheed, and United Space Alliance. 

NAV AIR: The Office of the Navy AIR Training Systems Division maintains a comprehensive list of 
publications authored by its researchers. Many of these publications describe the use of human factors 
tools in the areas of decision making and team training. 

DISA: The Defense Information Systems Agency manages the Human Systems Information Analysis 
Center (HSLAC). In addition to providing services, HSIAC also maintains a gateway to resources on 
human factors methods, tools, and techniques. 

FAA: The Federal Aviation Administration Technical Center contains links to many resources used by 
the FAA to integrate human factors into the design process of air traffic management systems and 
cockpits. There is considerable overlap between the needs of the FAA and NASA and the tools described 
in many of the FAA documents are useful sources for the NASA toolkit. 

Compendex*Plus: A general search for additional documents was conducted using the digital library 
resources at KSC and at Florida International University. These sources were used to identify additional 
tools as well as conduct more in depth analysis of the tools that were identified in the previous sources 
(see section 4 for more information on the tool analysis). 

3.2 Human characteristics addressed by HF Tools 

As the tools were compiled, they were differentiated according to the human characteristic(s) that each 
one addressed. Six major categories were identified. Each major category was further divided into 
subcategories to isolate the specific target of the tool. Table 3-2 presents the major categories and the 
subcategories contained within each one. 
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Table 3-1. SOURCES REVIEWED FOR HUMAN FACTORS TOOLS 


Source 

Description 

Reference 

DoD 

Compilation of human factors tools by the 
DoD - Human Factors Engineering 
Technical Advisory Group (HFE TAG) 

http ://dtica. dtic. mil/ddsm/ 

ONR 

Compilation of human factors tools by the 
Office of Naval Research Science and 
Technology - Manning Affordability 
Initiative (ONR S&T MAI) 

http :// www . manningaffordability 
com/S&tweb/Indexhse.htm 

Eurocontrol 

Compilation of human factors tools by 
Eurocontrol - Human Factors Integration in 
Future ATM Systems 

http ://www. eurocontrol.int/ 
eatmp/hifa/index.html 

NASA 

Compilation of human factors tools by 
NASA Headquarters - Office of Safety and 
Mission Assurance 

http://humanreliability-pbma- 

kms.intranets.com 

NAV AIR 

Publications from the Office of the Navy 
AIR - Training Systems Division 

http://www.ntsc.navy.mil/ 

DISA 

Tools and resources from the Defense 
Information Systems Agency - Human 
Systems Information Analysis Center 

http://iac.dtic.mil/hsiac/ 

FAA 

Information resources from the FAA 
Technical Center 

http://www.tc.faa.gov 

Compendex*Plus 

Library database of scholarly engineering 
publications 

http ://www . ei . org / eicorp/eicorp 


Physical: Tools that address physical requirements and limitations were divided into five categories: 
overexertion injury, repetitive trauma, fatigue, reach/fit, and toolkits. Overexertion injury refers to a 
musculoskeletal injury that is sustained as a result of an exertion that exceeds the capacity of the 
individual. These tools rely on biomechanical models of the musculoskeletal system to determine at what 
level of exertion users are likely to be injured. Some calculate a risk score indicating an estimate of the 
injury risk. Others present the quantitative values calculated by the model and rely on the analyst to 
interpret the data. 

Repetitive trauma refers to musculoskeletal injuries that result from the cumulative effects of many small 
exertions. These tools consider the major risk factors that have been linked to repetitive trauma injuries 
and assist the analyst in quantifying the degree of risk inherent in a system design. Some of these tools 
calculate cumulative risk scores that combine the risks identified from each risk factor. Others are 
checklists that simply highlight the risk factors that are present. Some of these tools provide 
recommendations based on the results to reduce the risk of repetitive trauma injury. 

Fatigue can affect system performance in two ways. Fatigued workers work slower and with reduced 
quality. This can reduce the efficiency or effectiveness of the system. Fatigue can also increase the 
likelihood of a user experiencing an injury due to overexertion or motor control error. Tools that address 
fatigue try to identify fatigue that results from either metabolic energy expenditure or lack of sleep. They 
can provide a risk score or present the metabolic levels and require the analyst to interpret them. 
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Reach and fit tools address the anthropometric characteristics of the human components of the system, the 
physical dimensions of the system layout, and the requirements of the human-system interaction. In 
general, reach/fit tools evaluate CAD layouts of the system, adding manikin representations of human 
users using a distribution of population dimensions to insure that any movement requirements are feasible 
given the physical requirements of the tasks. These tools can include evaluation of cones of vision, 
collision detection, analysis of clothing requirements for extreme environments, and other considerations. 

Toolkits provide a variety of analysis methods, generally assembled from the general literature but with a 
consistent interface and the ability to share data. The consistency allows multiple analyses to be 
conducted without repeated data entry and allow analysts to learn a single interface model. 

Information Processing: Tools that address information processing were divided into four categories: 
perception, situation awareness, decision making and mental workload. Perception tools evaluate the 
ability of the human user to perceive the physical characteristics of the system. Multiple modalities can 
be addressed, but generally include visibility, audibility, and tactility. Some tools also consider 
perception between team members for the purpose of coordinating activities and command & control. 

Situation awareness (S A) tools evaluate the accuracy of the user’s mental model of the important 
characteristics of the current state of the system and his or her ability to project that model into the future. 
There are a wide variety of situation awareness tool types. Query insertion tools ask the user relevant 
questions at periodic intervals to determine how aware of the important characteristics he or she is. Other 
tools use subjective scales, requiring users to use introspection to rate their awareness of the situation on a 
variety of dimensions. Some attempt to determine the SA using physiological characteristics. If a critical 
cue is displayed, responses such as event-related potential can be measured directly to determine if the 
cue was noticed. Performance-based tools correlate certain aspects of performance with SA. Finally, 
observer ratings have subject matter experts (SMEs) observe the user’s performance and measure the SA 
based on the appearance of specific behaviors. 

Decision making (DM) tools are used to investigate the details of the decision making process applied by 
users to identify modifications to the system, additional training, or decision support systems that can be 
applied to improve system performance. In general, a detailed interview must be conducted to identify 
the cognitive processes that lead to target behaviors. 

Mental work load refers to the amount of a user’s mental capacity that is occupied during system 
operation. These tools resemble SA tools in that they can use subjective scales, physiological measures, 
performance -based measures, or cognitive models. Mental workload tools have been customized to a 
variety of situations and there is a wide variety of tools available. 

Human Reliability and Error. Because of the extreme consequences that can result from human errors, 
there is a great need to measure human reliability and predict the types and probabilities of errors that 
may occur. Tools for evaluating human reliability address one or more stages of the analysis process 
listed in Table 3-2. In order to analyze human reliability, it is useful to analyze the tasks, identify all 
possible errors that can occur, analyze the consequences of the errors, and predict the likelihood that each 
error may occur. Error analysis tools can use cognitive models, operator interviews, incident reports, 
and/or statistical techniques. Some human reliability tools also assist the analyst in identifying ways to 
reduce the likelihood or magnitude of errors that occur. Usability tools are specifically designed to 
evaluate error types and probabilities in human-machine interfaces. 
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Manpower, Personnel, and Training: Manpower, personnel, and training (MPT) tools are used to 
determine the number of people required to achieve satisfactory performance from a system, what kinds 
of people are required in terms of skill sets, abilities and authority, and what training would improve 
system performance or reduce manpower requirements. 

Static Resources: There are many handbooks, guidelines, and standards that have been created within the 
Department of Defense and in general industry to address human factors requirements. Resources were 
obtained from US DoD, US federal agencies, European organizations, NASA, and private organizations. 

Comprehensive Modeling Tools: Comprehensive models attempt to describe all of human behavior. 
They can be used to predict actual behavior, test theoretical hypotheses, and model populations. For the 
toolkit, models that attempt to predict actual behavior were considered. Task network models do not 
explain why behaviors occur, but predict the times that activities require and probabilities of successful 
mission completion. Cognitive architecture models begin with first principles of human cognition and 
thus do attempt to explain the mechanisms of behavior. They can predict what behaviors are likely to 
occur and can be matched with task networks to predict time and reliability as well. Digital human 
models create simulations of physical aspects of humans to predict the most likely movements and paths 
taken by humans to accomplish tasks and can be linked with biomechanical models to predict injury risk. 

Table 3-2. MAJOR CATEGORIES OF HUMAN CHARACTERISTICS TARGETED BY 
HF TOOLS AND THE SUBCATEGORIES IDENTIFIED WITHIN EACH 


Physical 

Information 

Processing 

Human 

Reliability and 
Error 

Manpower, 
Personnel, and 
Training 

Static 

Resources 

Comprehensive 

Modeling 

Tools 

overexertion 

injury 

perception 

task analysis 

manpower 

handbooks 

task network 
models 

repetitive 

trauma 

situation 

awareness 

error 

identification 

personnel 

standards 

cognitive 

architecture 

models 

fatigue 

decision 

making 

error analysis 

training 

guidelines 

digital human 
models 

reach/fit 

mental 

workload 

error 

quantification 




toolkits 


usability 

evaluation 





4. TOOL ANALYSIS 

As each tool was identified, a detailed analysis was conducted to determine how it could be applied most 
effectively within the systems engineering process at NASA. Several factors were determined to be 
critical components as to whether a particular tool could be applied to a particular design effort. 

4.1 Expertise 

The expertise required to use the tool is an essential consideration. Tools differed significantly in the 
amount of knowledge of human factors fundamentals that is necessary to use them effectively. For 
example, modeling a task using a cognitive architecture model is impossible without considerable 
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expertise in cognitive engineering. On the other side of the spectrum, using a repetitive trauma checklist 
to evaluate a computer workstation can be accomplished with virtually no knowledge of human factors. 
Each tool falls along a continuum between these two extremes Performance-based mental workload 
tools are easy to implement without human factors expertise, but selecting a performance measure 
requires a familiarity with the components of mental workload and the types of tasks that draw ffom each 
mental resource. Systems engineers should not select tools that require more human factors expertise 
than is available on the design team. 

Another type of expertise that must be considered when selecting a tool involves use of the tool itself. 
Some tools can be used effectively after a short perusal of the manual. These tools have intuitive 
interfaces and/or simple processes. Subjective mental workload scales, risk-scoring repeated trauma 
tools, and many human error tools require little tool expertise. When selecting an unfamiliar tool, it is 
necessary to select one that does not require past experience to use effectively. Some vendors 
recommend training classes that can last up to 2 weeks. Projects operating within time constraints may 
not be able to take advantage of these tools. 

A third type of expertise is the familiarity with the requirements of the job being analyzed or designed. In 
some cases, the analyst can effectively use a tool while observing the end user perform the job without 
really understanding what is happening. Many of the physical tools fall into this category. However, 
others require intimate familiarity with how and why each step of the job is performed. Analysts who do 
not have access to experienced end users and are unfamiliar with the job under study must select a tool 
that can be implemented without such knowledge. 

For each of these types of expertise, the toolkit should provide some insight into how much is needed in 
order to accomplish the goals for which the tool is intended. In many cases, it depends on details of the 
project that do not generalize to all uses of a tool. However, the toolkit can provide guidance that helps 
an analyst determine if he/she has sufficient expertise in order to effectively use the tool for its intended 
purpose. 

4.2 Stage of Development 

Tools are generally developed for application at a particular stage of system development. Six phases of 
human factors application have been integrated into the NASA Systems Lifecycle Stages (see Figure 4-1). 
While gathering user requirements, tools can assist designers to understand what the needs of the system 
are, how previous versions have failed to meet expectations, and to identify potential constraints. Tools 
that address this phase cannot require analysis of specific design components. 

During conceptual design, tools can help designers compare alternative design approaches, interaction 
metaphors, high level architectures, and physical layout schemes. Physical tools that predict injury based 
only on general task requirments can be implemented here. Mental workload tools that estimate workload 
based on models of task dynamics and do not require SME interaction can also be useful. 

During detailed design, much of the interpersonal communications, interface designs, and workstation 
layouts are largely determined. Tools that evaluate display legibility, fault trees, and repetitive trauma 
checklists can be useful here. Tools that use high fidelity simulations to identify design constraints and 
bottlenecks can be implemented to significantly reduce the costs of redesign once physical mockups are 
completed. 
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Figure 4-1 . INTEGRATION OF HUMAN FACTORS INTO THE NASA SYSTEM LIFECYCLE 


After the design is completed and a physical system has been constructed, tools that involve analyzing 
user performance directly can be used to identify deficiencies before the system is launched. 
Performance-based workload measures, situation awareness tools, statistical error quantification tools and 
video-based physical tools are most effective to evaluate completed designs. 


Some tools are designed to monitor systems over time during use. At this phase, it is critical that tools are 
not intrusive so they do not impair user performance. This is critical both because it would invalidate the 
results and because it could sacrifice the effectiveness of the system during the evaluation. Instantaneous 
subjective workload scales, continuous safety sampling, and ergonomic workstation checklists are 
intended for in-use monitoring. 


Several tools, particularly those that determine the cause of human errors, are designed to be used during 
incident investigation. Examples include root cause analysis and cognitive task analysis. During the 
investigation process, access to users who were involved in the incident is usually required. 

4.3 Resources 


Selection of an appropriate tool must also consider the resources required to obtain sufficient accuracy 
and precision in the results. The most basic resource is cost. Tools that are owned by the federal 
government or are available at no cost should be selected over a tool with similar capability that costs tens 
of thousands of dollars. The range of tool prices varies considerably. The toolkit lists prices for fixed 
price tools and facilitates the request for quotes for tools that are flexibly priced. 

Time is often a more constrained resource than money. Tools vary in the amount of time required to 
complete various types of analyses. Often the time requirements are proportional to the scope of the 
analysis. For example, fault trees that address every aspect and component of a design take considerably 
longer than an identical fault tree focused on just a partial set of components. However, the toolkit can 
provide general insight into the time requirements for analysis at various levels and provide comparisons 
among tools that provide similar analysis. 

4.4 Access 

The toolkit can also facilitate access to the tool and tool-related support materials. I .inks to tool vendor 
contact information, web sites, and training registration can facilitate getting started once a tool has been 
selected. Links to independent validation studies and tool analyses such as can be found at the ONR 
Manning Affordability Initiative web site can be very helpful during tool selection. Contact information 
for experienced users at NASA can help in the selection process as well as during use of the tool. 
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5. CONCLUSIONS 


In order for NASA to design systems that successfully consider human capabilities, it is essential that 
human factors is integrated into the systems engineering process. The inclusion of human factors can be 
greatly improved through a utility that assists systems engineers in the selection and application of human 
factors tools The IHFT is an effort to provide this assistance. The toolkit allows systems engineers to 
retrieve a set of human factors tools, along with advice on implementation of each tool. The selection 
criteria and advice are customized for the project, including its stage of development, resources available, 
and the expertise of the design team. Use of the IHFT will improve the safety, reliability and 
performance of systems throughout NASA. 
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